Method and Apparatus for Constructing Efficient Slepian-Wolf Codes With Mismatched Decoding

ABSTRACT

Disclosed is a method for constructing Slepian-Wolf codes, wherein the designed Slepian-Wolf codes are robust to mismatched decoding. The disclosed method for constructing Slepian-Wolf codes includes the steps of: choosing representative probability distributions from a set of possible probability distributions; choosing a probability distribution as a decoding metric; converting the chosen decoding metric to a cyclic-symmetric channel; computing the initial message value given the cyclic-symmetric channel; computing a set of probability distributions of the initial message given the initial message values and the representative probability distributions; optimizing the degree distribution given the set of probability distributions of the initial message; optimizing the decoding metric.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present application generally relates to methods and apparatuses forconstruction of codes for data-compression and, more particularly, to amethod and apparatus for constructing efficient Slepian-Wolf codes formismatched coding.

2. Background Description

Slepian-Wolf codes refer to codes used for compressing source data at anencoder, where the decoder has access to side information not availableto the encoder. There are many applications of these codes. Examples ofsuch applications include, but are not limited to, low complexityencoding of media, distributed source coding for sensor networks,scalable video coding, etc.

FIG. 1 shows a generic Slepian-Wolf coding system. The signal X^(n)=X₁X₂. . . X_(n), termed the Slepian-Wolf source 100, is compressed by theencoder 101 using a Slepian-Wolf code, the output of which is a set ofsymbols termed syndromes 102. These syndromes represent the compressedsource signal. The decoder 103 receives a signal Y^(n)=Y₁Y₂ . . . Y_(n),termed the Slepian-Wolf side information 104, which is used to decodethe signal X^(n) from the syndromes. Without loss of generality, it canbe assumed that X_(i) and Y_(i) take values in discrete sets {0, 1, . .. , M−1} and {0, 1 . . . , N−1}, respectively, for all i=1, 2, . . . ,n. The efficiency of compression (measured by the bit rate of thesyndromes required to decode X successfully) depends, among otherfactors, on the joint distribution of X_(i) and Y_(i) (denoted byP_(XY)), the goodness of the Slepian-Wolf code employed, and thedecoding method. The better the Slepian-Wolf code, the lower is thenumber of syndrome bit required for successful decoding for a givenjoint distribution and a decoding method.

Previous methods for designing Slepian-Wolf code are reported by S. S.Pradhan and K. Ramchandran, “Distributed source coding using syndromes(DISCUS): design and construction”, IEEE Transactions on InformationTheory, pp. 626-643, March 2003, J. Garcia-Frais and Y. Zhao,“Compression of correlated binary sources using turbo codes”, IEEECommunication Letters, 5:417-419, October 2001, V. Stankovic, A.Liveris, Z. Xiong, and C. Georghiades, “On code design for the generalSlepian-Wolf problem and for lossless multiterminal communicationsnetworks”, IEEE Transactions on Information Theory, pp. 1495-1507, April2006, J. Bajcsy and P. Mitran, “Coding for the Slepian-Wolf problem withturbo codes”, IEEE Globecom. pp. 1400-1404, November 2001, A. Aaron andB. Girod, “Compression with side information using turbo codes”,Proceedings of the IEEE Data Compression Conference (DCC), pp. 252-261,April 2002, A. D. Liveris et al., “Compression of binary sources withside information at the decoder using LDPC codes”, IEEE CommunicationLetters, vol. 6, pp. 440-442, 2002, J. Chen, D. He and A. Jagmohan,“Slepain-Wolf code design via source-channel correspondence”,Proceedings of IEEE International Symposium on Information Theory, July2006. These methods are developed under the assumption that the jointdistribution P_(XY) is perfectly known and the decoder uses a decodingmetric that is matched to P_(XY).

FIG. 2 shows an exemplary conventional method for constructingSlepian-Wolf codes. The input 200 to the design method is theprobability distribution P_(XY). The joint distribution 200 is firstconverted to a cyclic-symmetric channel distribution 201, i.e., Q_(V|U),with input alphabet {0, 1, . . . , M−1} and output alphabet {0, 1, . . ., M−1}×{0, 1, . . . , N−1}. The channel Q_(V|U) is related to P_(XY)through the equation V=(U+_(M) X,Y), where U is independent of (X,Y)whose distribution is P_(XY), and +_(M) denotes modulo-M addition. Thecyclic-symmetric channel Q_(V|U) is then used to compute the initialmessage value 202, i.e., m₀(v), through the equation:

${{m_{0}(v)} = \lbrack {{\log \frac{Q_{V|U}( v \middle| 1 )}{Q_{V|U}( v \middle| 0 )}},{\log \frac{Q_{V|U}( v \middle| 2 )}{Q_{V|U}( v \middle| 0 )}},\ldots \mspace{11mu},{\log \frac{Q_{V|U}( v \middle| {M - 1} )}{Q_{V|U}( v \middle| 0 )}}} \rbrack},{v \in {\{ {01,\ldots \mspace{11mu},{M - 1}} \} \times \{ {0,1,\ldots \mspace{11mu},{N -}} }}$

The initial message value m₀(v) and the probability distribution P_(XY)are used to determine the probability distribution of m₀ (V), i.e.,P_(m) ₀ 203 by setting the probability distribution of V equal toP_(XY). Given P_(m) ₀ , the density evolution means 204, which consistsof the density evolution algorithm and the standard degree distributionoptimization methods, outputs an optimized degree distribution 205. Theconstruction of Slepian-Wolf codes given the degree distribution isstraightforward.

However, for real applications, it is impossible to obtain a completelyaccurate estimation of the joint probability distribution P_(XY). Let{circumflex over (P)}_(XY) denote the estimated probability distributionthat is used by the decoder to recover the source signal X^(n) given thesyndromes and the side information Y^(n). For the case in which{circumflex over (P)}_(XY) is different from P_(XY), the decoder ismismatched to the true probability distribution (and therefore, thedecoding is referred to as mismatched decoding). For practicalSlepian-Wolf coding systems, it is essential that the imperfectknowledge of P_(XY) is taken into account when designing Slepian-Wolfcodes. It is also important to choose the optimal decoding metric in themismatched decoding case, where the decoding metric is the jointprobability distribution used for decoding. However, there are nopreviously known computationally feasible methods for constructing goodSlepian-Wolf codes with mismatched decoding and choosing the optimumdecoding metric.

Therefore, a need exists for an improved method for Slepian-Wolf codedesign wherein mismatched decoding resulting from the imperfectknowledge of joint probability distribution is taken into account. Therealso exists a need for an improved method for choosing the optimaldecoding metric in the mismatched decoding case.

SUMMARY OF THE INVENTION

It is an object of this invention to improve methods for constructingefficient Slepian-Wolf codes with mismatched decoding.

Another object of the present invention is to provide a method forchoosing the optimal decoding metric when the true probabilitydistribution is not completely known.

These and other objectives are attained with a new method for designingSlepian-Wolf codes. The method includes the steps of: choosingrepresentative probability distributions from a given set of possibleprobability distributions; computing a set of initial messageprobability distributions given the representative probabilitydistributions and a fixed decoding metric; using density evolution toobtain an optimized degree distribution given a set of initial messageprobability distributions; optimizing the decoding metric so that theresulting degree distribution yields the lowest syndrome bit rate.

The preferred embodiment of the invention provides a method forconstructing Slepian-Wolf codes with mismatched decoding. It allows thedesign of efficient and robust Slepian-Wolf codes for a prescribed setof probability distributions and a fixed decoding metric. The preferredembodiment of the invention also provides a method for choosing theoptimal decoding metric when the true probability distribution is notcompletely known.

The key advantage of the present invention is that it can be used toconstruct Slepian-Wolf codes that are robust to mismatched decoding.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, aspects and advantages will be betterunderstood from the following detailed description of a preferredembodiment of the invention with reference to the drawings, in which:

FIG. 1 is a block diagram illustrating the Slepian-Wolf coding method;

FIG. 2 is a block diagram illustrating a conventional Slepian-Wolf codedesign system;

FIG. 3 is a block diagram illustrating an embodiment of the presentinvention for constructing Slepian-Wolf codes with mismatched decoding;and

FIG. 4 is a block diagram illustrating a data compress and decompressionsystem based on the present invention.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT OF THE INVENTION

Embodiments of the present invention disclosed herein are intended to beillustrative only, since numerous modifications and variations of theseembodiments will be apparent to those of ordinary skill in the art. Inreference to the drawings, like numbers will indicate like partscontinuously throughout the views.

FIG. 3 depicts an exemplary embodiment of an apparatus which implementsthe method for constructing efficient Slepian-Wolf codes with mismatcheddecoding and choosing the optimal decoding metric. The inputs consist ofa set of possible probability distributions 300, i.e., P, and a decodingmetric set 301, i.e., {circumflex over (P)}. The output consists of adegree distribution 302 and a decoding metric 303, i.e. P*_(XY).

The key component of the method is a Slepian-Wolf code generator 304 formismatched decoding with two inputs: a set of representative probabilitydistributions 305 (e.g., P_(XY) ⁽¹⁾, P_(XY) ⁽²⁾, . . . , P_(XY) ^((m)))from P and a tentative decoding metric {circumflex over (P)}_(XY) 306from the decoding metric set {circumflex over (P)}. The decoding metric{circumflex over (P)}_(XY) is first converted to a cyclic-symmetricchannel 307, i.e., {circumflex over (Q)}_(V|U), with input alphabet {0,1, . . . , M−1} and output alphabet {0, 1, . . . , M−1}×{0, 1, . . . ,N−1}. The channel {circumflex over (Q)}_(V|U) is determined by{circumflex over (P)}_(XY) through the equation V=(U+_(M) X,Y), where Uis independent of (X,Y) whose distribution is {circumflex over(P)}_(XY), and +_(M) denotes modulo-M addition. The cyclic-symmetricchannel {circumflex over (Q)}_(V|U) is then used to compute the initialmessage value 308, i.e., {circumflex over (m)}₀(v), through theequation:

${{{\hat{m}}_{0}(v)} = \lbrack {{\log \frac{{\hat{Q}}_{V|U}( v \middle| 1 )}{{\hat{Q}}_{V|U}( v \middle| 0 )}},{\log \frac{{\hat{Q}}_{V|U}( v \middle| 2 )}{{\hat{Q}}_{V|U}( v \middle| 0 )}},\ldots \mspace{11mu},{\log \frac{{\hat{Q}}_{V|U}( v \middle| {M - 1} )}{{\hat{Q}}_{V|U}( v \middle| 0 )}}} \rbrack},{v \in {\{ {01,\ldots \mspace{11mu},{M - 1}} \} \times \{ {0,1,\ldots \mspace{11mu},{N -}} }}$

The initial message value {circumflex over (m)}₀(v) and the probabilitydistribution P_(XY) ^((i)) are used to determine the probabilitydistribution of {circumflex over (m)}₀(V), i.e.,P_({circumflex over (m)}) ₀ ^((i)) (output 309) by setting theprobability distribution of V equal to P_(XY) ^((i)) (i=1, 2, . . . ,m). Given P_({circumflex over (m)}) ₀ ⁽¹⁾, P_({circumflex over (m)}) ₀⁽²⁾, . . . , P_({circumflex over (m)}) ₀ ^((m)), the density evolutionmeans 310, which consists of the density evolution algorithm and thestandard degree distribution optimization methods, outputs an optimizeddegree distribution 311 such that the decoding error probability forevery initial message distribution P_({circumflex over (m)}) ₀ ^((i))(i=1, 2, . . . , m) converges to zero under density evolution.

The representative probability distributions are chosen from P by adistribution selector 312. Only a few illustrative selection methodswill be given here, since numerous modifications and variations will bestraightforward to those of ordinary skill in the art. In the case whereP is characterized by some parameters, the set of representativedistributions can be chosen by sampling the parameters. For example, ifP is a set of probability distributions {P^((λ))} parameterized by λwith λ between 0 and 1, then P⁽¹⁾ and P⁽²⁾ may be chosen as therepresentative probability distributions for P. In general, the numberof representative probability distributions depends on the desired levelof accuracy as well as the computational complexity constraints. In thecase where P is a finite set, the set of representative probabilitydistributions could be P itself.

The rate of degree distribution 311 outputted from the Slepian-Wolf codegenerator 304 is calculated using a rate computer 313, and is stored ina rate buffer 314.

The decoding metric set {circumflex over (P)} could be different from P.The size of {circumflex over (P)} can be chosen according to thecomputational complexity constraint. The tentative decoding metric{circumflex over (P)}_(XY) is chosen from {circumflex over (P)} by adecoding metric selector 315. The selection can be based on the rates inthe rate buffer. In the case where {circumflex over (P)} is a finiteset, the decoding metric selector can use exhaustive search to optimizeover {circumflex over (P)}_(XY) so that the resulting degreedistribution yields the lowest syndrome bit rate. The decoding metricselector can also use gradient search. Modified selection rules based onother search methods are obvious to those of ordinary skill in the art.

Finally, the optimum {circumflex over (P)}_(XY) is output as thedecoding metric 303, i.e., P*_(XY), and the associated degreedistribution 302. The construction of Slepian-Wolf codes given thedegree distribution is straightforward to those of ordinary skill in theart.

FIG. 4 shows an exemplary embodiment of a Slepian-Wolf data compressionand decompression system using the presented invention. Given a set ofpossible probability distributions 400 and a set of decoding metrics401, the presented new Slepian-Wolf code design outputs a degreedistribution 402 and a decoding metric 403. The data compressor 405encodes the source data 404 using the degree distribution 402 (moreprecisely, the data compressor 405 encodes the source data 404 using theSlepian-Wolf code induced by the degree distribution 402) and sends thesyndrome 406 to the data decompressor 407. The data decompressor 407reconstructs the data source 409 using the syndrome 406, the decodingmetric 403 and the side information 408.

While the invention has been described in terms of a single preferredembodiment, those skilled in the art will recognize that the inventioncan be practiced with modification within the spirit and scope of theappended claims.

1. A method for data compression comprising the steps of: constructing aSlepian-Wolf code with one or more inputs wherein mismatched decodingresulting from imperfect knowledge of joint probability distribution istaken into account; encoding a source data using the said Slepian-Wolfcode to output a syndrome data; and decoding the source data from thesyndrome data and a side information data by choosing an optimaldecoding metric for the mismatched decoding.
 2. The method for datacompression according to claim 1, further comprising the step ofselecting the decoding metric as the output decoding metric of the stepof constructing the Slepian-Wolf code.
 3. A method for constructingSlepian-Wolf codes with one or more inputs, wherein mismatched decodingresulting from imperfect knowledge of joint probability distribution istaken into account, comprising the steps of: a) choosing a tentativedecoding metric from one of the inputs; b) selecting representativeprobability distributions from one of the inputs; c) converting thechosen decoding metric to a cyclic-symmetric channel; d) computing a setof probability distributions of the initial message given the initialmessage value and the representative probability distributions; e)optimizing the degree distribution given the set of probabilitydistributions of the initial message; and f) generating a Slepian-Wolfcode according to the said degree of distribution.
 4. The methodaccording to claim 3, wherein the inputs include a set of probabilitydistributions and optionally a set of decoding metrics.
 5. The methodaccording to claim 3, further comprising the steps of: g) calculating arate of said degree distribution in step e) and storing the rate in arate buffer; h) selecting a new decoding metric from one of the inputsgiven the stored rates in said rate buffer; and i) repeating steps b)through g) until a prescribed stopping criterion is met.
 6. The methodaccording to claim 5, wherein the outputs include a degree distributionand optionally a decoding metric.
 7. The method described in claim 5wherein the said prescribed stopping criterion is the absolutedifference between two successive rates of said degree distributioncalculated in step e is less than a given threshold.
 8. The methodaccording to claim 4, wherein the said set of possible probabilitydistribution is obtained through prior experiments.
 9. The methodaccording to claim 8, wherein the said prior experiments includeestimating the parameters of probability distributions from a parametricclass.
 10. The method according to claim 4, wherein the said set ofpossible decoding metrics is obtained through prior experiments.
 11. Themethod according to claim 10, wherein the said prior experiments includeestimating the parameters of probability distributions from a parametricclass.
 12. The method according to claim 6, wherein the said degreedistribution is the degree distribution of low-density parity-checkcodes.
 13. The method according to claim 3, wherein for a fixed decodingmetric, the said degree distribution is optimized by using densityevolution and/or other standard channel code design tools.
 14. Themethod according to claim 5, wherein the said decoding metric isoptimized so that the resulting degree distribution yields the lowestsyndrome bit rate.
 15. The method according to claim 14, wherein theselection in step g) is done by exhaustive search for a finite set ofdecoding metrics.
 16. The method according to claim 14, wherein theselection in step g) is done using gradient search.
 17. The methodaccording to claim 3, wherein the set of representative probabilitydistributions is the same as the set of a finite set of possibleprobability distributions.
 18. The method according to claim 3, whereinthe set of representative probability distributions is selected bysampling the parameters of the set of possible probability distributionswhich is characterized by some parameters.
 19. An encoding apparatus forsource data which constructs Slepian-Wolf codes with two inputs and twooutputs comprising: input means for receiving a tentative chosendecoding metric belonging to a given decoding metric set; selectionmeans for selecting representative probability distributions from a setof possible probability distributions; conversion means for convertingthe chosen decoding metric to a cyclic-symmetric channel; computationmeans for computing a set of probability distributions of an initialmessage value of the source data and the representative probabilitydistributions; optimization means for optimizing a degree distributiongiven a set of probability distributions of the initial message; and anencoder for encoding a source data using Slepian-Wolf code induced bythe degree distribution to output a syndrome in a data compressionsystem.
 20. The encoding apparatus according to claim 19, furthercomprising: output means for outputting a decoding metric; and datadecompression means receiving side information and said decoding metricfor reconstructing the data source from the syndrome.